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ABSTRACT 

The instruction aanual provides 12 step-by-step 
instructions for air traffic control supervisors in conducting 
over-tna-shoulder training observations of enroute center 
controllers. Since the priaary purpose of the review is to guickly 
identify training needs and requireaents, the control 
responsibilities are approached fro» a deficiency viewpoint r and 
descriptions of functions and duties are written as perforiance 
deficiencies. The functions and duties covered are: separation, 
control judgaent, traffic lanageaent, operating aethods and 
procedures, coordination and coaaunication, phraseology, and 
equipaent. These are further subdivided into specific tasks. It is 
recoaaended that the controller's perforaance be observed for a 
ainiaua of four 10-ain^te periods. The Review Fori for recording 
observations is included. (AG) 
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SECTION 1 
INTRODUCTIOW AND OVERVIEW 

This document provlttes step-by-step instructions for air traffic control 
supervisors in conducting over-tlie-shoulder training observations of enroute 
center controllers. By following the procedures detailed in this document, 
performance eureas in which the controller has deficiencies will be identified. 
Analysis of the observation results will help to determine the specific 
training required to improve individual controller performance. 

The primary purpose of this over-the-shoulder training review is to quickly 
identify the training needs and requirements of controllers by observing 
any operating deficiencies firsthand during control performance. There- 
fore, the control responsibilities are approached from a deficiency viewpoint, 
and descriptions of functions and duties are written as performance deficiencies. 

The responsibilities of the enroute uanter controller Joave been analyzed to 
arrive at a set of functions and duties directly related to technical per- 
formance in controlling aircraft. This set of technical functions has been 
categorized into seven proficiency or skill areas. Each proficiency or skill 
area contains a number of duties related to that control skill. This document 
contains descriptions of the fimctions and duties, aiid provides examples of 
the types of possible observed deficiencies. The descriptors are provided as 
typical indicators and are not to be construed as the only items for deter- 
mining deficiencies. 

This document also contains a detailed set of procedures for conducting the 
over-the-shoulder training review. In addition to observing controller 
performance^ the reviewer is required to determin'* a ccwpleKity level for the 
conditions under which the control is performed. Also, the highest amount of 
simultaneous traffic handled during the training review must be determined. 
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Instructions for these and other routine procedures are explained in the 
instructions portion of this document. 

The review form contains 27 control duties within the seven proficiency or 
skill areas. Dependent on the situation, some of these control duties c<.n 
be considered more important or more significant than others and, therefore, 
more critical to the task of controller performance. The asterisks (*) on 
the review form identify these more significant duties wherein deficiencies 
indicate possible immediate training needs. 

Knowledge of the descr< tions of duties and the detailed step-by-step 
instructions for use are the only requirements for conducting an observation 
and completing the over-the-shoulder review form. 

This over-tha-shoulder training review is designed to allow team super- 
visors a rapid overall review of controller technical performance. As a 
training needs indicator, it is not designed to replace the semi-annual 
evaluation process. However, the results may be used to supplement the 
semi-annual evaluation process, fls a supervisory aid, soma of the resultant 

benefits from use are: 

1. Quickly identifies controller control deficiencies. 

2. Enables determination of training needs. 

3. Can be conducted in short time period. 

4. Can be used in making performance comparisons. 

5. Can be used to determine training progreso. 

6. Reduction of time for identifying training needs allows 
more time for other supervisory duties. 

Other long-range benefits include the use of the statistical data to project 
possible deficiency levels with changes in traffic load or complexity factors. 
With improved methods of application and subsequent use of the information, 
other benefits can be added to the list. This instrument will be as useful 
as the individual supervisor permits, based upon his desire for improvement 
in the areas of evaluation and training. 
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SECTION 2 

TRAINING INDIOVTOR DESCRIPTIONS 

FUNCTIONS AND DUTIES 

The primary purpose of this over-the-shoulder training review method is to 
identify controller training requirements by observation of deficiencies 
during performance. To assist supervisors, the controller functions and 
duties have been described based on a deficiency definition. The following 
provides descriptions of the functions and duties to be observed during 
over-the-shoulder training reviews. As there is some overlap in the sub- 
jective definition of proficiency areas, detailed descriptors or typical 
examples of deficiencies are also provided for each of the duties to be 
reviewed. 

The descriptions are not all encoB^)assing nor are they meant to establish 
limits to the duties to be reviewed. Experienced supervisory reviewers may 
consider other factors and may note and mark these in descriptions or on the 
review form as part of the observed deficienciei. 

The functions ami duties contained in the following section are sequenced 
in the same order on the over-the-shoulder review form to assist in using 
the descriptions. 
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SEPARATION 

1. Separation is not ensured ? Does not provide control instructions or 
restrictions to ensure separation standards are maintained at all times* 

e Does not issue restrictions to prevent loss of separation » 

• Does not issue control instructions to prevent loss of separation* 

• Does not ensure separation standards are maintained under broad*- 
band, narrowband, or manual control* 

CONTROL JUDGMENT 

2. Awareness is not maintained ; Continuous attention is not provided to all 
facets of the control environment to ensure that discrepancies do not 
exist; does not take action to resolve any discovered discrepancies; and 
does not ensure that other personnel/facilities affected by these dis- 
crepancies are notified. Fails to develop an accurate ''pictiure in his 
mind" of the existing traffic, and does not display the ability to fore- 
cast future positions and relationships of aircraft* 

m Does nou control in a manner to ensure future separation or flow 
problems will not exist? generally only reacts to situations that 
are eminent* 

• Fails to ensure flight strip information is in agreement with 
aircraft route/position* 

• Does not verify if a discrepancy exists anytime an unusual or non- 
standard situation occurs* 

m Is not aware of subsequent controller requirements* 

• Doesn't pay attention to what other controllers are doing* 

• Uses control procedures which result in xmnecessary workloads and 
stress being placed on other controllers/facilities* 
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3, Poor control judgment i» applied t Xssuea control inatructions or res- 
trictions that are not correct or reasonable. Does not use control pro- 
cedures to allow for efficient flow of traffic. Fails to obtain all 
available facts about the control situation , does not fully and accurately 
determine aircraft requirenents » and does not carefully plan procedures 
prior to issuing instructions. 

• Pails to ensure awareness of all facts concerning the control 
situation. 

• Doesn't determine the requirements of aircraft. 

• Has to interrupt passing of clearances or instructions to obtain 
necessary information. 

• Xssues unnecessary restrictions. 

• Accepts handof fs of aircraft that are in conflict or are about to 
los® separation. 

• Utilises incorrect vector procedures. 

• Incorrectly evaluates the factors affecting safety of aircraft. 

• Incorrectly evaluates the factors affecting traffic flow. 

• Does not inform aircraft and appropriate personnel of existing 
significant situations. 

• Controls in a manner which allows significant situations to develop 
resulting in system errors, lessening of safety, or unnecessary 
slowing of traffic. 

• Allows unneeded separation. 

• Allows unnecessary altitude, heading or speed changes. 

• Mas to issue additional instructions because previous actions or 
decisions were incorrect. 
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4. Control a ctions are incxxrr&ctly planned : Does not control traffic within 
area of responsibility in an orderly manner. 

• Control procedures are not orderly and tend to cause unnecessary 
complexity and confusion. 

• Applies flow control procedures earlier than required resulting in 
SLXi unnecessary slowdown of traffic flow. 

TRAFFIC MANAGEMENT 

5. Positive control of situation is not provided ; Controller does not take 
command of control situations; acts in a hesitant and unsure manner. 

• Acts vmsure of himself 

6. Prompt action to correct errors is not taken : Fails to recognize when 
an error exists or has been made, and therefore does not take prompt 
action to correct the error. 

m Does not recognize when incorrect information has been passed to 
aircraft or other ATC personnel. 

• Does not act rapidly to correct the error. 

7. Sector overload not prevented when possible : Fails to observe present 
and forecast traffic to predict if an overload may occur, and therefore 
does not take appropriate actions to prevent or lessen an overload 
situation* 

• Does not recognize potential overload situations. 

• Fails to initiate action to prevent or limit the overload. 
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8. Aircraft identity is not painf ined i Pails to dotemine aircraft position 
through appropria^^ use of beacon, turn procedures or coordination with 
other sectors/facilities I does not maintain positive identification 
during the entire tixae the aircraft is within the sector area of re- 
sponsibility. 

e Employs incorrect beacon or radar procedures* 

• uses shrimp boats incorrectly. 

• Does not keep data blocks in association with correct targets, 
e Fails to reidentify aircraft when a doubt exists. 

9. Board management is not maintained ; Does not keep the flight strip bay 
properly organized. Does not adequately monitor flight data information 
and the position of aircraft on the display equipment. 

e Fails to closely monitor all air traffic data. 

• Unnecessarily moves to other positions. 

e Engages in conversations about miscellaneous matters when the 

control situation dictates concentration on the control situation. 

e Incorrectly sequences the flight strip. 

• Does not remove deadwood. 

OPERATING METHODS MID PROCEDURES 

10. Flight strip postings are incomplete or incorrect ? Does not post all 
r<£quired information on flight strips. 

• Fails to post altitude changes » estimated times » radar contact, etc. 
e Posts data in incorrect area of strip. 

• Does not use reqpsirtd colors. 

• Postings are not legible. jLO 
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11. Clearance delivery is incorrect/incomplete/untimely : Does not issue clear- 
ances in the correct format or order; is not specific and uses incorrect 
phraseology, 

• Does not use specific terms to describe a fix* 

• Uses inappropriate clearance fixes. 

• Exceeds clearance authority. 

12. Letters of Agreement/center directives not adhered to i Performance of 
control functions and duties are not in compliance with agreed upon faci- 
lity procedures^ handbook^ or center directives. 

13. Necessary traffic advisories are not provided : Does not follow the required 
format when passing traffic information # and does not pass traffic advisories 
when workload permits. 

# Fails to provide traffic advisories to aircraft in PCJi on workload 
permitting basis. 

# Pails to provide advisory service to all radar identified aircraft. 
m Does not issue complete traffic information in the required format, 

14^ Navigational assistance is not provided : Fails to issue appropriate control 
inscructions to avoid significant situations, to obtain operational advan- 
tages to avoid congested areas, or to provide shorter routes. 

# Does not provide n^.vigational assistance when an operational advantage 
to the pilot or controller would be gained. 
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15. Handoff procedures are incorrectly performed : Does not verify aircraft 
position or coordinate any restrictions or special instructions? uses 
incorrect interphone procedures. 

• Does not verify aircraft position. 

• Does not satisfactorily coordinate special restrictions or instructions. 

• Fails to correctly update flight strips. 

• Does not perform handoff at appropriate time/position. 

COORDINATION AND COMKUNICATIQM 

X6, Professional manner is not maintained ; Does not danonatrate the ability 
to think clearly and act rapidly during stressful situations? and does 
not convey a positive and professional in^ression at all times. 

• Fails to rem&in calm under stress. 

• Does not continue to control in an effective manner when the control 
situation becomes stressful. 

• Does not convey the impression of a skilled professional who can 
successfully handle the situation to other controllers, pUots# and 
related personnel. 

Coordination is not thorough ? Fails to coordins^-A.^l information that 
is pertinent to the situation. Fails to ensure that personnel receiving 
information h*ve all of the contents, or does not acknowledge all infor- 
mation passed to him. 

• Does not ensure tha^. all required information has been passed. 

• Does not verify or acknowledge all information exchanges. 
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18. CoBflnunication is unclear » not concise t Does not ensure that all data 

passed or received is understood; has to repeat information using different 
words to convey the intended meaning. 

• Does not make intentions knovm. 

• Does not ensure information is understood. 

• Has to repeatedly explain intentions. 



PHRASEOLOGY 



19. Standard Phraseology is not adhered to ; Pails to use words and phrasing s 
in accordance with the requirements of the duty being performed. 

• Does not use approved procedural vfordsr phrases and formats. 
m Improvises when not necessary. 

20. Uses poor voice quality t Quality of the controllers voice is such that it 
is hard to understand speech. 

• Poor voice pitch or timbre. 

• Voice is harsh, grating, or causes lack of understanding. 

21. Speech rate is incorrect : Other personnel have difficulty understanding 
what has been said because controller speaks too rapidly or too slowly. 

22. Makes unnecessary transmissions ? Transmits information over radio or inter- 
phone that is not required? or transmits information as separate messages 
when it would be more efficient to combine information. 

• Unnecessary use of radio or interphones. 
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EQUIPHENT 

23. Equipment status in£onn»tion is not maintained ; 'oes not maintain know- 
ledge o£ sector equipn^t operating st&tus or NAVAZDS used by aircraft 
flying through sector; and does not report malfunctioning equipment as 
required. 

• Fails to determine status of equipment performance. 

• Does not report malfunctions. 

24. Computer entries are incorrect : Does not display knowledge of the entries 
required when interacting with the coc^uter. 

• Makes too numy entry errors. 

• Fails to make necessary computer entries which results in required 
control data not being displayed. 

• Continually having to repeat ccanputer switch actions. 

25. ftdjustmfent of control display is incorrect t Pails to adjust the radar 
presentation, whether narrowband or broadband, to present the best display 
possible; does not recognize when maintenance assistance is necessary; and 
does not ensure that radar control is provided when display has deteriorated 
beyond safe limits. 

• Pails to enter all required data into computer for required sector 
display under ROP. 

• Does not display entire sector. 

• Adjusts gain so that targets are fuzzy and not sharp. 
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26. Equipment capabilities are not fully utilized/understood : Does not 
utilise .available equipment to the fullest extent possible; does not display 
knowledge of capabilities and limitations of equipment and its associated 
backup. 

• Uses other than best equipment available. 

m Is not aware of built-in delay in radio equipment. 

• Is unaware of secondairy equipnent when primary fails « 

27. Radar changeover is incorrectly performed ; Pauls to follow orderly proce- 
dures to ensure a smooth transition from one radar processing system to 
another. 

• Fails to take correct switch actions. 

• Does not ensure radaur displays are presented on appropriate scopes. 

• Performance of changeover between sector personnel is neither orderly 
nor coordinated. 
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SECTIQW 3 



TRAINING BgVISW INSTRUCTIQMS 

Following are the detailed procedures for conducting over-the-shoulder training 
reviews using the Center Controller Over-The-Shoulder Training Review Form. The 
controller's performance should be observed for a minimum of four 10-minute 
periods (40 minutes) . The form has space for five review periods and additional 
forms may also be used if more than the minimum of four reviews is necessary. 

Administrative information required on the form such as name, date, and facility, 
may be completed ionediately prior to or subsequent to the re^-iews, but should 
not be entered during the conduct of a review period. 

The supervisor/reviewer should review descriptions of possible deficiencies 
specified in Section 2, as well as the following training review instructions 
prior to the start of the review session. The sector to be used in conducting 
reviews should be determined by considering the types of traffic and 
complexity factors as well as the traffic load. 

After plug-in at the appropriate monitor position, the review is conducted by 
using the following procedures. 

1. select a period of time during the watch when traffic will be 
adequate (preferably moderate or higher) to allow observation of 
most duties. 

2. The review form allows for five review periods. Plan to take at 
least four reviews (40 minutes total) during the session, although 
more may be taken if desired? use additional forms, if necessary. 
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3. Each review consists of a 10-minute, closely monitored observation 
of duties. At the end of 10 minutes you may pause to catch your 
breathe reevaluate the situation, or vAiatever. Dependent on the 
traffic load, you may either delay the next review or begin a new 
review immediately. Continue taking observations until you have 
covered at least four review periods (40 minutes total) . 

4. When the observation starts, note the time in the left-hand 
portion of the space marked Time at the top of the review column. 

5. During the review, keep track of the amount of traffic handled by 
the controller. The highest amount of simultaneous traffic (peak 
traffic) for the review should be noted in the Peak Traffic Count 
box at the top of the review column. 

6. If the controller is observed incorrectly performing a duty, or 
is deficient in the performance of a duty, place a dot in the 
appropriate duty box to indicate each observed deficiency. 

7. Each review should end after a 10-minute period. Note the end time 
in the right-hand portion of the space marked Time at the top 

of the review column. 

8. Consider the nature of traffic complexity for the 10-minute review 
period and enter the designator in the appropriate box at the top 
of the review column. (The ccanment side of the review form con- 
tains a traffic complexity code for various levels of complexity.) 

9. After completion of the review session, carefully count all the 
dot training indicators. Place the area totals in the space pro- 
vided after each skill area. The overall dot training indicator 
total for the review is placed at the botton of the column. 

10. Duties 1 through 4 have been asterisked to indicate that defi- 
ciencies noted for these control duties are considered more 
significant than deficiencies for the other duties. 
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IX. Technical performance on some duties may not be observed during 
the review and therefore, the duty box will be blank. For these r 
place an X (no action) in the duty box. This will serve to dis- 
tinguish the no action items as opposed to acceptable performance 
items (remaining blank boxes) . 

12. Finalize the review by entering pertinent conments and other 

information on the comment side of the review form. All required 
information should be completed in the spaces provided. 

The completed form provides a handy reference to the review period and furnishes 
details of deficiencies or weaknesses observed. It also serves as a supervisory 
aid in conducting the post review interview with the controller. As suchr it 
helps to identify the training needs of the controller for discussion purposes. 
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CENTER CONTROLLER 
OVER-THE-SHOULDER TRAINING REVIEW 



CONTROLLER; 



FAClLII V: 



DATE: 



POSITION R£V)EV¥EOt 



AREA: 



StCTO*^ 



TRAFFIC COMPL-CXITY 

R - ROUTINE. NOT DIFFICULT 
O ~ OCCASIONALLY DIFFICULT 
M ^ MOSTLY DIFFICULT 
V - VERY DIFFICULT 



REVIEW NO. 

DEFICIENCY COUNT 
PEAK TRAFFIC 
COMPLEXITY 



COMMENTS REGARDING THIS REVIEW SESSION 



REVIEW 



REVIEW 2: 



R£Vl£W 3: 



REVIEW 4i 



REVIEW Bt 



DISCUiSION or REVIEW RESULTS 


CONTROLLER: 


iDATE: 

i f 


REVIEIIVER*. 


1 
1 

. L 
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OVER-THE-SHOULDER REVIEW FORM 

PEAK TRAFFIC COUNT 

COMPLEXITY INDICATOR 
TIME 

SEPARATION 

#1. Se^wtation is not unsured. 

TotHl deficiencies observed this aiea 
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CONTROL JUDGMENT 

*2 Awareness is not maintained. 
*3. Poor control judgment is applied. 
*4. Control actions aro incorrectly planned. 
Total deficiencies observed this area 



TRAFFIC MANAGEMENT 

5 Positive control uf situation is nut provided. 

6. Prompt action to correct errors is not taken. 

7. Sector overload not prevented when possible, 

8. Aircraft identity is not maintained. 

9. Board management is not maintained. 
Total deficiencies observed this area 




OPERATING METHODS AND PROCEDURES 

10. Flight strip postings are incomplete or incorrect. 

11. Clearance delivery is incorrect/mcomplete/unttmely. 

12. Letters of Agreement/center directives not adhered to. 

13. Necessary traffic advisories are not provided. 

14. Navigational assistance is not provided. 

15. Handoff procedures are mcDnectly performed. 
Total deficiencies observed this area 



COORDINATION AND COMMUNICATION 

16. Professionai manner is not maintained. 

17. Coordination is not thoiough. 

18. CommunicstiC'^" is ur^clfc'ar, not concise, 
Total deficiencies observed this area 



PHRASEOLOGY 

19. Standard phraseology is not adhered to 

20. Uses poor voice quality. 

21. Speech rate is incorrect. 

22. Makes unnecessary transmissions. 
Total deficiencies observed this area 
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EQUIPMENT 

23. Equipment status information is not maintained. 

24. Computer entries are incorrect. 

25. Adjustment of control display is incorrect. 

26. Eq'ilpment capabilities not fully utilized/understood, 

27. Radar changeover is incorrectly performed. 
Total deficiencies observed this area 

Total deficiencies observed this review 
•Critical Training indicator 
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